Visual speech recognition and utterance segmentation based on mouth movement

This paper presents a vision-based approach to recognize speech without evaluating the acoustic signals. The proposed technique combines motion features and support vector machines (SVMs) to classify utterances. Segmentation of utterances is important in a visual speech recognition system. This research proposes a video segmentation method to detect the start and end frames of isolated utterances from an image sequence. Frames that correspond to `speaking' and `silence' phases are identified based on mouth movement information. The experimental results demonstrate that the proposed visual speech recognition technique yields high accuracy in a phoneme classification task. Potential applications of such a system are, e.g., human computer interface (HCI) for mobility-impaired users, lip-reading mobile phones, in-vehicle systems, and improvement of speech-based computer control in noisy environments

Subtle hand gesture identification for HCI using temporal decorrelation source separation BSS of surface EMG

Hand gesture identification has various human computer interaction (HCI) applications. This paper presents a method for subtle hand gesture identification from sEMG of the forearm by decomposing the signal into components originating from different muscles. The processing requires the decomposition of the surface EMG by temporal decorrelation source separation (TDSEP) based blind source separation technique. Pattern classification of the separated signal is performed in the second step with a back propagation neural network. The focus of this work is to establish a simple, yet robust system that can be used to identify subtle complex hand actions and gestures for control of prosthesis and other HCI based devices. The proposed model based approach is able to overcome the ambiguity problems (order and magnitude problem) of BSS methods by selecting an a priori mixing matrix based on known hand muscle anatomy. The paper reports experimental results, where the system was able to reliably recognize different subtle hand gesture with an overall accuracy of 97%. The advantage of such a system is that it is easy to train by a lay user, and can easily be implemented in real time after the initial training. The paper also highlights the importance of mixing matrix analysis in BSS technique

Dibenzyl 3,3′,4,4′-tetra­methyl-5,5′-(ethynedi­yl)bis­(pyrrole-2-carboxyl­ate)

The title mol­ecule, C30H28N2O4, has crystallographic twofold rotation symmetry, with the pyrrole planes forming a dihedral angle of 40.49 (4)°. The pyrrole N—H donor and adjacent ester carbonyl acceptor form R 2 2(10) hydrogen-bonded rings about inversion centers, leading to chains of hydrogen-bonded mol­ecules along [001]

ICA based identification of sources in sEMG

The identification of number of active muscles during a complex action is useful information to identify the action, and to determine pathologies. Biosignals such as surface electromyogram are a result of the summation of electrical activity of a number of sources. The complexity of the anatomy and actions results in difficulty in identifying the number of active sources from the multiple channel recordings. ICA has been applied to sEMG to separate the signals originating from different sources. But it is often difficult to determine the number of active sources that may vary between different actions and gestures. This paper reports research conducted to evaluate the use of ICA for the separation of bioelectric signals when the number of active sources may not be known. The paper proposes the use of value of the determinant of the global matrix generated using sub-band ICA for identifying the number of active sources. The results indicate that the technique is successful in identifying the number of active muscles for complex hand gestures. The results support the applications such as human computer interface

Performance comparison of ICA algorithms for isometric hand gesture identification using surface EMG

There is an urgent need for developing a robust technique that can identify small and subtle hand and other body movements with applications in health, rehabilitation and defence. Surface electromyogram (sEMG) is a measure of the electrical activity of the muscles and a measure of the strength of muscle contraction. While this may be a good measure of the actions and gestures, this is unable to identify small variations in the muscle activity, especially when there are number of simultaneously active muscles. Independent component analysis (ICA) is a statistical based source separation technique that has been shown to be suitable for the decomposition of signals such as sEMG and been shown to improve the ability of sEMG to identify small variations in muscle activity. ICA algorithms using multivariate statistical data analysis technique have been successfully used for signal extraction and source separation in the field of biomedical and statistical signal processing. Recent research has resulted in the development of number of different ICA technique. While there are some researchers who have compared their techniques with the existing methods for audio examples, there is no comparison of performance between ICA algorithms for biosignal applications such as surface electromyography (sEMG) applications. With ICA being the feasible method for source separation and decomposition of biosignals, it is important to compare the different techniques and determine the most suitable method for the applications. This paper has studied the performance of four ICA algorithms (FastICA, JADE, Infomax and TDSEP) for decomposition of sEMG to identify subtle hand gestures. Comparing several ICA algorithms, it is observed that an algorithm based on temporal decorrelation method (TDSEP) which is based on the second order statistics gives the best performance

Unspoken vowel recognition using facial electromyogram

The paper aims to identify speech using the facial muscle activity without the audio signals. The paper presents an effective technique that measures the relative muscle activity of the articulatory muscles. Five English vowels were used as recognition variables. This paper reports using moving root mean square (RMS) of surface electromyogram (SEMG) of four facial muscles to segment the signal and identify the start and end of the utterance. The RMS of the signal between the start and end markers was integrated and normalised. This represented the relative muscle activity of the four muscles. These were classified using back propagation neural network to identify the speech. The technique was successfully used to classify 5 vowels into three classes and was not sensitive to the variation in speed and the style of speaking of the different subjects. The results also show that this technique was suitable for classifying the 5 vowels into 5 classes when trained for each of the subjects. It is suggested that such a technology may be used for the user to give simple unvoiced commands when trained for the specific user

Reliability and variability in facial electromyography for identification of speech and for human computer control: an experimental study

The need for developing reliable and flexible human computer interface is increased and applications of HCI have been in each and every field. Human factors play important role in these kinds of interfaces. This research investigates the use of facial muscle activity for a reliable interface to identify voiceless speech-based commands without any audio signals. We propose a method of measuring the relative activity of the articulatory muscles of the face for recognition of unvoiced vowels. System performance and reliability were also tested for the case of variations like inter-subject, inter-day, and different languages. In these investigations, English vowels and German vowels were used as recognition variables. The designed methodology used linear and non-linear classification based on statistical clustering techniques and artificial neural network architecture. The results show that there is a variability in facial muscle activation during vowel utterance between different subjects, different days. These results will be helpful in use of facial electromyography for identification of speech and in other application such as human computer control

Facial muscle activity patterns for recognition of utterances in native and foreign language: testing for its reliability and flexibility

The need for developing reliable and flexible human computer interface is increased and applications of HCI have been in each and every field. Human factors play an important role in these kinds of interfaces. Research and development of new human computer interaction (HCI) techniques that enhance the flexibility and reliability for the user are important. Research on new methods of computer control has focused on three types of body functions: speech, bioelectrical activity, and use of mechanical sensors. Speech operated systems have the advantage that these provide the user with flexibility. Such systems have the potential for making computer control effortless and natural. This chapter summarizes research conducted to investigate the use of facial muscle activity for a reliable interface to identify voiceless speech based commands without any audio signals